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ABSTEACT^ 

ThXB paper is addressed to administrators and 
.evaluators in school districts whiah are initiating bilingual 
education programs in coinpliance with a D*S^ Supreme Court decision* 
Major e¥aluation problems are described and remeaies suggested* Each 
step in the seguence^ from needs assessment through final report^ is 
discussed and illustrated* The objsotive is to share Milwauk^ long 
e^parience (since 196S) in bilingual program evaluatipn with other 
districts, and to forearm by foreMarning* (Author) 
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Many states have mandated bilingual/^iQultural education prosTSms in 
sdhool districts having a mijdjnmn number of non-aigliah-dominmt ^pilss* These 
laws require little reporting. Howeverj Patriots which apply for federal 
fundd.ng must write a proposal which includes an evaluation pl^* If accepted, 
the evaluation plan will have to be implementaid, from seleetion of participants 
throu^ data collection a^d reporting, Ah*^ there the rub I Many are the 
pitfalls between proposal and report, purpose is to fore-arm researchers 
new to bilingual evaluation by describing problems encountered since 1969 ii, 
the evaluation of ffl.lwaukee*s Title VII Bilin^al/bieultiiral Education Program, 
I will take you quickly tihrough the typical evaluation steps and point out 
these characteristics of eachs the Ideal situation, the Reality, and some 
Problems which result because of the discrepancy between ideal and real* I 
will then humbly advise you of a Saving Grace and a Remedy . 



lift 1) Ne eds. Proposal, and Evaluation Desim 

The first steps deal with a needs analysis, the establishment of goals &nd 
"Objectives, and the_vfri ting of a proposal* Ideally, the goals amd objectives 
represent a consensus of parents and ttaehers, arrived at throu^ lengthy 
..^^^ study and discussion. In Reality, the proposal deadline often requires a 
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haaty effort to deserlba iraiovative courses of study which have scarcely 
advanced beyond the idea stage* The evaluator eneounters his first Problem 
here for the proposal oiust include m evaluation plm and timeline, yet aseess- 
ment instnanents have not been located or developed*. M evaluation plan con- 
ceived in hasta and a timeline, deyoloped mthout knowledge of course content 
just won't work. The Saving Grace Qmt the discrepancy diaappears once the 
Rensedy baa been applied. The RemedkT is edification of the evaluation dasi^ 
and timeline by poBt-proposal eonfereneea vdth supervisors and teachera^ 

2) Selection of Farticipanta . , 

The pupil population is the evaluator *s basic concern in trying to deter- 
mine -Uie effect of an innovative program* Ideally, pupils who wish a bilingual 
education would be randomly assign&d to program and non-program classea. In 
Reality, all eligible applicants are enrolled. This creates a Problem in 
clawing comparison groups* Vhere eligible pupils are not enrolled, the com- 
parison group is luilike the treatment group in its support of bilingual educa- 
tion and of tan contains a larger proportion of Englieh-dominMLt pupils* These 
situations force extra caution in drawing inferences from the results of cosi- 
parative data analyses* The Saving Grace is that longitudinal data within the 
program group rc^eal^ learning patterns of value in curriculiam planning. The 
only Remedy I can envision would be ^mding guidelines mandating the controlled 
studies which cannot be required within local school system political realities* 
No one wants to deny a pupil the opportunity to enroll in an educational pro- 
gram* 



3) Tdentifi cation of Dominarit Lanyua^e 

The pupil' a first Isuipiage is used as a bridge to teaohing the second 
language, to develop readinesB reading skills, md to teaeh subjeot con- 
tent. With EDonolingualSj, firet laai^age identifieation Is no problem* However, 
many bilinguals interehange languages to the extent that the teaeher o^mot 
clearly identify the domnmt Imiguage. Ideally^ ttiere would be -tests of 
language dominanoe at all grade levels. In Reality , . most valid instruments 
have been nomed on kindergarten and first-grade pupils. The Problem is that, 
beyond the age of six, iM^guage dominance is a matter of teaGher Jud^ient and 
subjeqt- to error in the oaae of bilinmial chilfl^en. The Sa^^d^ Graoe is that 
most bilingual/bioulti^al teachers have micoimion perGeption for -identif ioation 
©f the dominant language.. Tlie Remedy is development of valid instrimente 
extending the norais beyond early chilttoQod* 



^) ]&istrumentation 

to a developmerital programp students are taught all subjedt oontent in two 
languages from kindergarten throu^ twelfth grade* Ideally,, there would be an 
availablt pool of standardised aDhievement teats with equivalent Qiglish/Spanish 
foms, Aeimally, such test develo^ent has been limted. toe must use local 
or published translations of English tests. In either oasei editing is 
neoessa^ because of vocabulary differenoes among "Uie Spanish*apeaking natinns. 
Lack of time for local test construction md validation and the absence of 
large normative or comparison groups are the ^obl ems, . The Saving Grace is 
that Remedies for Spanish reading are being developed via New York City's 



"Language Assessmtnt Batteries", S^*s "Sober Espamol", Dr, Oscar Ozete'e 
Tests of Spanish Reading Comprehension » Phi Ifelta Kappa has developed various 
Spanish/feigllsh achievement tests for Grades 7 to 12. m most other curriculum 
^ areas, the Problem is with us but the Saving Grace and Reinady have yet to 
appear. 

5) Monitoring 

^ Ideallyj pupil progress would be assessed periodieally in bo-Ui languages* 
Development of a monitoring system from Kindergarten through Grade 12 in - two 
languages in several sohools is a Problem* TOie Reality is that ayetematie 
monitoring is not possible vathout one or more IHill-time evaluators. 

In a monitoring system^ the greatest Problems are teacher resist^ce to the 
time and paperwork involved and the pereeived threat of the monitor process* 
A Saving Grace is that teachers leara much about their pupils through testlngj 
and supervisors can pinpoint areas where assistance with program planning Is 
needed. The Remedy for monitoring problems appears to lie in improving the 
system over time and in experlenQing its benefits* . 

6) ^ Surveys 

Students, staff, atoinlstrators, parents, and community are surveyed 
annually. Their suggestions are solicited for use in planing a^ more effective 
program* ^ Student and staff response is good. Ideally, parent/coimiimity 
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response would also be high* In Reality, these groups have a low rate of 
return, The Problein is that parent/co^mmlty input is given great weight in 
progr^ planning but the input Is based on a small representation of the popi- 
lation, Ohere is no Saving Grace for lack of parent input* A Reraedy may lie 
In the establishment of a parent/connwnity group to work vdth the evaluator to 
en courage^ greater parent/comiunity response* 

7) Data ' ^Pr o c e g e ing 

Ideally^ assessment instruments would be saored electronically and analyzed 
by computer* Actually, the pupil population ie too small md tiiie evaluation 
deei^ too complex for investment in maoh eoi^uter processing. Most of the 
work is done by hand using a desk calculator* The Problem here is operator 
error which imst be closely monitored. The Sa\dng Grace is limited assist^ce 
by data processing personnel* The Remedy woiild be additional searetarial, 
progr^ner^ and computer time, all unli&l|^at this point* 

8) Data Analysis 

Ideally, comparisons could be made between large program and comparison 
groups* Actually, the comparison population is small. Frequently i data need 
to be analysed according to language dominance (e.g-i pfrforaimce on a test of 
English reading) which results in very small groups nested within a class or 
grade level. The reralting Problem is the difficulty of achieving confidence 
in 'the interpretation of results." mie Saving Grace is that a longitudinal 



study can reveal trends over tirae. The Remedies would appear to be randona 
aesignment of pupila to treataent ajid control groups and the pooling of data 
from aeveral sohools as the program ejqpands* 

9) Annual Report 

Ideally I the content of the ^mual evaluation report would be a factor in 
prograin pl^iningi proposal writing, and funding* The Reality ia that plaming, 
proposal writing, and funttng precede posttesting, data analysis, and report 
witlng* The Problem is that aometimes evaluation outoomea sipial a need to 
change dire et ion in program planning i for ex^^le, the develo^ent of alters 
native ourriciila for students whose reading does not improve as a resiilt of 
enrollment in ^e B5.1ingual Reading Center* A Saving Grace is the fact ttat 
the finings result in eventual action but it seems a loss of precious time for 
which there^ seems to be no Remedy* 

10) Language 

Ideally, the evaluator, like the staff, should be a linguist. In Reality, 
few researchers are bilin^al* Neither are we epqperts in any of the othier 
academic disaiplines with which we work. Rather, armed >rtth some toiowledge 
about leaminfi, himiaji behaviori md statistical teehniquee we, like Renaissance 
Man, absorb much of what we work with* ^ir ijaitial naivete would be a eertain 
problem had we not the Saving Grace of cooperative educators to work vrtifch* 
Bilingual staff members provide, a Saving Grace by willingly lending their 



expertise to the evaluator. The training of bilingual evaluation specialists 
is the Remedy, 

Ten processes ond a series of problems have been itemized* Moat of tha. 
processes, but only a sMple of problems, were identified* CMr Research 
Departnient Direotor, Dwight Rowe, defines a problem as "an opportoiity"* May 
you be blessed with many a Saving Grace and Remedy for the Problems whieh mil 
be your opportunity in evaluating bilingual progr^s. 



